What and how do different stakeholders contribute to intervention development? A mixed methods study.

Background: UK Medical Research Council guidelines recommend end-user involvement in intervention development. There is limited evidence on the contributions of different end-users to this process. The aim of this Study Within A Trial (SWAT) was to identify and compare contributions from two groups of end-users - people with diabetes’ (PWD) and healthcare professionals’ (HCPs), during consensus meetings to inform an intervention to improve retinopathy screening uptake. Methods: A mixed method, explanatory sequential design comprising a survey and three semi-structured consensus meetings was used. PWD were randomly assigned to a PWD only or combined meeting. HCPs attended a HCP only or combined meeting, based on availability. In the survey, participants rated intervention proposals on acceptability and feasibility. Survey results informed the meeting topic guide. Transcripts were analysed deductively to compare feedback on intervention proposals, suggestions for new content, and contributions to the final intervention. Results: Overall, 13 PWD and 17 HCPs completed the survey, and 16 PWD and 15 HCPs attended meetings. For 31 of the 39 intervention proposals in the survey, there were differences (≥10%) between the proportion of HCPs and PWD who rated proposals as acceptable and/or feasible. End-user groups shared and unique concerns about proposals; both were concerned about informing but not scaring people when communicating risk, while concerns about resources were mostly unique to HCPs and concerns about privacy were mostly unique to PWD. Fewer suggestions for new intervention content from the combined meeting were integrated into the final intervention as they were not feasible for implementation in general practice. Participants contributed four new behaviour change techniques not present in the original proposals: goal setting (outcome), restructuring the physical environment, material incentive (behaviour) and punishment. Conclusions: Preferences for intervention content may differ across end-user groups, with feedback varying depending on whether end-users are involved simultaneously or separately.


Introduction
According to the UK Medical Research Council guidance on the development and evaluation of complex interventions, interventions should be developed with user involvement, drawing on existing evidence and appropriate theory 1 . User involvement usually includes those who will deliver the intervention (often healthcare professionals [HCPs]) and the intended target population (often patients and the public). It is expected to improve the intervention fit with the target group's perceived needs enhancing acceptability; feasibility; evaluability and adoption 2,3 .
While some studies have found that different end-users have similar priorities and preferences when making decisions about health research and service delivery 4,5 , other studies have found that different end-users endorse different perspectives 6,7 . In the context of intervention development, limited evidence exists on what different intervention users contribute to the process. Morton et al. have suggested that different stakeholders may have different priorities for intervention content 8 . For instance, the cost of a proposed intervention might be more important than feasibility for intervention commissioners, whereas those receiving the intervention may be more concerned with its acceptability. However, more substantive research is needed to empirically examine and compare what different end-users contribute to the intervention development process.
Furthermore, group dynamics are complex, and some user groups may find it more difficult to voice their priorities and perspectives compared with others 9 . Studies involving end-users in intervention development tend to treat all end-users (e.g., patients and HCPs) as one homogenous group 10-12 . We previously compared participants' experiences of taking part in meetings to inform the development of an intervention to increase diabetic retinopathy screening attendance 13 . Three meetings were held comprising people with diabetes only; a combined meeting of people with diabetes and HCPs; and a HCP only meeting. We found that involving both people with diabetes and HCPs in the same group led to a perceived lack of common ground where both groups felt undervalued by the other group and were reluctant to express their opinions 13 . While these findings might suggest that intervention end-users may find it more acceptable to involve each group separately, we are also keen to know whether their contributions during these meetings differed according to group composition. Understanding whether user contributions differ according to group composition could enable researchers to design and conduct more appropriate and effective user involvement activities which in turn could potentially improve intervention fit with the target group's perceived needs.
The aim of this Study Within A Trial (SWAT) was to identify and compare people with diabetes' and HCPs' contributions during

Amendments from Version 1
We would like to thank the reviewers for their suggestions and comments to improve the academic merit of our research. We have addressed each on a point-by-point basis in the responses section. Main amendments made to the paper include: a. we accept that in most interventions, patients or service users should be considered 'key players that everyone else has a stake in' , however, this current intervention was a multilevel intervention which targeted both people with diabetes and healthcare professionals, that is, it had components that targeted people with diabetes (i.e., personal testimonials, reminders, information provision etc.) and professionals working in general practice (i.e., audit, feedback, electronic prompts etc.). It was made clear at the outset of the meetings that the focus was both people with diabetes and HCPs. We agree that this positioning likely influenced how PWD (and HCPs) contributed during the combined meeting. We aim to reflect this in the discussion on our previous analysis of both PWD and HCP experiences of taking part in the consensus meetings. We also agree with the suggestion that the researchers were an 'invisible power' in the decision-making process, who influenced the final intervention and have added a paragraph to the discussion section to address this.
b. In terms of PPI methodology, we have discussed the reviewer's suggestion to clarify the distinction between research participants and PPI contributors and have re-written the sentences in question. In terms of PPI involvement in this study, GF was involved throughout the research process as has been correctly pointed out. We also consulted with an existing PPI group on the design of the consensus meeting invitation letter, evidence summary and self-completion survey. We have added further information to the methods section to give a more accurate depiction of the role of PPI in this study.
Any further responses from the reviewers can be found at the end of the article REVISED three consensus meetings to inform intervention development, including their feedback on the acceptability and feasibility of intervention content, suggestions for new intervention content, and contributions to the final intervention.

Methods
This SWAT was embedded in the intervention development phase of the Improving Diabetes Eye-Screening Attendance (IDEAs) pilot trial 14 . IDEAs used a systematic three-step process combining theory, user involvement and evidence on intervention effectiveness to develop a multifaceted intervention targeting people with diabetes and HCPs to improve uptake of RetinaScreen, a national Diabetic Retinopathy Screening (DRS) programme 15 . As part of the user involvement process, three semi-structured consensus meetings were conducted to review and discuss proposals for intervention content.

Design
This SWAT is a mixed method study using an explanatory sequential design 16 . Quantitative data (self-completion participant survey) were collected and analysed first, followed by the qualitative data (consensus group meetings) which were collected and analysed second in sequence 17 . The quantitative results provided an overview of participant ratings of acceptable and feasible intervention content, while the qualitative analysis allowed for further exploration of why participants rated intervention content the way they did by using a topic guide informed by survey findings.

People with diabetes
People with diabetes were recruited using an information flyer developed by the research team including a graphic designer (http://doi.org/10.5281/zenodo.4321202). The flyer was distributed using a range of recruitment strategies including social marketing recruitment, community outreach recruitment, health system recruitment, and partnering with other organisations. All individuals who contacted the study team and returned a short demographic survey (Supplementary File 1 in the Extended data 18 ) were randomly assigned (using an online random number generator) to either the meeting for the people with diabetes only, or the combined meeting.

Health care professionals
HCPs were recruited through local professional networks known to the study team. An email invitation was sent to 50 HCPs (practice nurses, diabetes nurse specialists, general practitioners, and specialist physicians). All HCPs were allocated based on their availability to the HCP-only meeting or combined meeting. Further details on the recruitment process have been described in detail elsewhere 13 .

Quantitative phase
Before each consensus meeting, participants were sent an evidence summary of barriers to and enablers of attendance at diabetic retinopathy screening, and interventions to address non-attendance (Supplementary File 2 in the Extended data 18 ), and a self-completion survey (Supplementary File 3 in the Extended data 18 ). The evidence summary and survey were designed with input from the Irish National Adult Literacy Agency and a Patient and Public Involvement (PPI) group and revised based on their feedback.
The survey outlined 39 proposals for intervention content that were grouped at the practice-level ('ways to encourage the practice staff to make sure person attends') and patient-level ('ways to encourage the person to attend diabetes eye screening'). The proposals contained operationalised behaviour change techniques (BCTs), defined as an "observable, replicable, and irreducible components of an intervention" that have the potential to change behaviour 19 . The proposals (operationalised techniques) were short statements/descriptions of how the selected BCT would be put into practice 20 , in line with the study focus on increasing diabetic retinopathy screening uptake. The BCTs in the survey were selected to address known barriers to and enablers of screening attendance based on previous formative research conducted by the IDEAs research team 15 and existing evidence of their effectiveness either in interventions to increase retinopathy screening attendance or interventions in other settings 21,22 . A total of 24 unique BCTs were operationalised across the 39 intervention proposals in the survey. Further details on these 24 BCTs has been provided in Supplementary File 4 in the Extended data 18 .
In the survey, participants were asked to rate the acceptability and feasibility of each proposal. All items were rated on a Likert response scale ranging from 1 to 5 (from 'strongly disagree' to 'strongly agree') with higher scores indicating greater acceptability or feasibility. These survey questions were adapted from existing measures developed by Weiner et al. to rate implementation acceptability and feasibility 23 . Acceptability was defined as the perception among end-users that the intervention proposal was agreeable or satisfactory. Feasibility was defined as the extent to which the intervention proposal could be successfully implemented in general practice. People with diabetes received a paper format of the survey while HCPs received an electronic format.

Qualitative phase
Following completion of the surveys, participants took part in one of three consensus group meetings. Each meeting was held for two hours in University College Cork and was facilitated by the same facilitator experienced in consensus group techniques/processes. This facilitator was a male professor of health services research who held no relationship with participants. This individual was a member of the Project Steering Group, acting in an advisory capacity but not actively involved in data collection and analysis beyond the consensus meetings. This individual was invited to facilitate the meetings as they could adopt a neutral position having no vested interest in any of the intervention components.
During the meetings, a summary of the ratings of acceptability/ feasibility was presented to participants. This was followed by a series of small group discussions (facilitated by members of the research team) where participants were asked to discuss how each intervention proposal would work in practice (See Supplementary File 5 in the Extended data 18 for Facilitator Guide). Facilitators asked participants to discuss and give feedback on both practice-level and patient-level proposals. Prompts about patient-level proposals included 1) who should deliver the message to remind patients to attend diabetes eye screening? 2) how should the message be delivered? 3) when should the message be delivered? and 4) what should the message contain? Participants were asked to focus their discussion on proposals where the consensus on acceptability and feasibility based on the survey was unclear. However, given the semi-structured nature of the meetings, participants also made new suggestions. The small group discussions and the feedback to the larger group were digitally audio recorded with participant consent.

Data analysis
Participant survey responses were entered into SPSS software (version 26, RRID:SCR_016479) and analysed using descriptive statistics. Consensus meeting transcripts were analysed using NVivo 12 software (RRID:SCR_014802). If this software were unavailable, it would be possible to conduct the analysis using Excel and Word.

Comparing end-users' feedback on the acceptability and feasibility of intervention content
To examine participants' ratings of the acceptability and feasibility of intervention proposals, the five-point Likert scale used in the survey was collapsed into three categories: 'disagree' [1 strongly disagree, 2 disagree], 'neither disagree or agree' [3] and 'agree' [4 agree, 5 strongly agree]. Contingency tables were generated for each intervention proposal by participant type (HCP or people with diabetes) and Fisher's exact test was used as appropriate 24 . Results were examined to identify proposals which had a difference (≥10%) between the proportion of HCPs and people with diabetes who agreed that intervention proposal was feasible and/or acceptable.
Guided by the survey results, interview transcripts were analysed using deductive content analysis. A codebook (developed a priori by LOM) designed to mirror the self-completion survey to identify and code feedback on specific proposals was used. Participants in the combined meeting were asked to reach group consensus on intervention proposals, therefore it was difficult to attribute feedback exclusively to people with diabetes or HCPs or both. Therefore, the people with diabetes only meeting and the HCP only meeting were analysed before the combined meeting was analysed, to allow the researchers to see whether feedback from the combined meeting echoed that of the people with diabetes only and HCP only meetings.
To compare participants' feedback on the acceptability and feasibility of intervention proposals, thematic analysis was performed by LOM, guided by joint displays of the survey results and qualitative coding. The joint displays were examined for recurring patterns between survey ratings and discussion during the consensus meetings, to identify reasons for agreement/disagreement e.g., what was or was not acceptable/feasible to whom, and why. An overview of this sequence of mixed methods is provided in Figure 1.

Comparing end users' suggestions for new intervention content
To identify and compare end-users' suggestions for new intervention content, two researchers (ER and FR) conducted a deductive content analysis 25 to identify suggested changes to proposed intervention content and suggestions of additional intervention content. Both researchers read the consensus meeting transcripts multiple times (data familiarisation) and then independently extracted all suggestions made by participants in relation to intervention content and mode of delivery. A suggestion was defined prior to data analysis as any suggestion about intervention content or mode of delivery proposed by a member of the group, at any stage during the meeting, that was agreed with by one or more other members of the group. Agreement or disagreement between participants was ascertained based on explicit verbal expression or sounds or noises which conveyed their agreement or disagreement (e.g., mmm). The two researchers met to discuss the suggestions they had extracted. Any differences were discussed, and agreement was reached by consensus on the list of suggestions put forward by participants. Each new suggestion was then coded (yes/no) according to whether it would be feasible to incorporate into the intervention to be delivered. The scope of the intervention was defined as: • purpose of the intervention (to improve the uptake of a national DRS service) • intervention setting (general practice in Ireland) • timeline (2 years to develop and test the feasibility of the intervention) • budget (the IDEAs study was providing a practice participation fees plus some materials/consumables approx. €1,000 per practice) • practice resources (each practice needed to have at least one practice nurse and computerised patient records) To identify how each new suggestion aligned with existing behavioural change techniques, they were mapped to Behaviour Change Technique Taxonomy (BCTTv1) 26 . Further information on how this mapping was conducted is provided in Supplementary File 6 in the Extended data 18 .

Comparing end users' contributions to the final intervention
Using deductive content analysis, one researcher (ER) categorised (yes/no) all recommendations (including feedback on proposals and suggestions for new intervention content) according to whether they were incorporated into the final intervention. Full details about the decision process regarding the final intervention content has been published elsewhere 15 . The final decision on the intervention content was made by a subgroup of the IDEAs study research team and a GP collaborator, basing decisions on the APEASE (affordability, practicability, effectiveness, acceptability, side effects, equity, sustainability) criteria. Practicality and acceptability criteria were populated based on findings from the rating survey and the discussions during the consensus meetings. The effectiveness criterion was based on a rapid evidence review of different approaches to improve screening uptake. Remaining criteria (affordability, equity, side-effects (unintended consequences), sustainability) were based on group discussions about what was feasible, bearing in mind previous formative research with patients and healthcare professionals and organisational factors relating to the primary care environment.

Patient and Public Involvement (PPI)
A PPI contributor (GF) was involved in the SWAT from the outset. GF is a person with diabetes, previously known to the lead author (ER). She contributed to the initial discussions about the study which ultimately informed the SWAT grant application, reviewed the grant application prior to submission and made changes to its content including the addition of disseminating the research amongst people with diabetes. GF was also involved in the development of materials used to recruit people with diabetes and assisted the research team with recruitment by posting recruitment flyers online via social media networks. She contributed to and reviewed each draft of this manuscript and is a co-author on this publication. The lead author also worked with a separate primary care research PPI group to develop and refine the materials that were sent to participants prior to the consensus meeting. PPI contributors in this group were asked to review draft versions of the consensus meeting invitation letter, evidence summary and self-completion questionnaire. Significant changes were made to the wording and layout of the materials as a result of their input. For example, section headings were added to the selfcompletion questionnaire which reduced its length from five pages to three pages. After the consensus meetings were conducted, the IDEAs study worked with a dedicated PPI group throughout the duration of the trial 15 .

Ethical approval
The study received ethical approval from the Social Research Ethics Committee (SREC) at University College Cork (Log number 2018-122, approval received 13/08/2018). Written informed consent was obtained from all participants prior to completing the rating survey and taking part in the consensus meetings.

Results
Comparing end users' feedback on the acceptability and feasibility of intervention content In total, 30 participants (13 people with diabetes and 17 HCPs) completed and returned the surveys. Missingness within the data ranged from 3.3% to 6.7%, depending on the survey proposal.
There was incomplete data for 6 participants (4 people with diabetes, 2 HCPs). Table 1 presents the 31 proposals which had differences (≥10%) between the proportion of HCPs and people with diabetes who agreed the proposal was acceptable and/or feasible 18 .  Table 1. Patient-level and practice-level proposals with survey results and related concerns identified from the consensus meetings.

Concerns about intervention content
Following integration of the survey results and qualitative feedback from the consensus meetings, themes related to the preference for and several main concerns about acceptable and feasible intervention content (Figure 2). Table 1 presents where these relate to intervention proposals and whether it was a joint concern, or preference, of both people with diabetes and HCPs, HCPs only or people with diabetes only.
The results are organised according to the joint preference, joint concerns, HCP concerns and people with diabetes' concerns. Examples of intervention proposals that relate to each area of concern are presented, along with the survey results and a short summary of participants' feedback from the consensus groups.

Joint preference
-Some people with diabetes have a limited understanding of the need for and practicalities of screening Participants in all three meetings considered several intervention proposals to be acceptable and feasible because they believed some people with diabetes have a limited understanding of the screening process. In the survey, both people with diabetes and HCPs agreed the proposal to use someone in the practice who would explain the difference between routine eye checks and the screening test was acceptable (92.3% vs. 94.1%, respectively), though they differed in agreement with feasibility (75% vs. 94.1%, respectively). Data from the meetings provided no indication as to why people with diabetes rated feasibility lower than HCPs, however both groups flagged that there is confusion among some people with diabetes about the difference between routine eye tests and retinal screening. Participants in the combined meeting agreed that messages delivered to patients should outline the difference between routine eye tests and retinal screening and emphasise that damage can be asymptomatic to dispel the "false sense of security". Similarly, participants in the people with diabetes only meeting thought messages should aim to increase patient understanding of the screening process. For example, highlighting the possible consequences of non-attendance and "alert you (people with diabetes) to the dangers involved". Participants in the HCP only meeting agreed messages should emphasize that screening is free.
In the survey, less people with diabetes than HCPs agreed the proposal to arrange practical support was acceptable (66.7% vs 82.4% respectively), though less HCPs agreed it was feasible (58.3% vs HCPs 41.2%). Participants in the people with diabetes only meeting felt many people with diabetes are not aware of the need to organise transportation for after the screening procedure, and so messages should tell people they would need support rather than arranging it for them. HCPs had concerns about the feasibility of this proposal, which are discussed below under the concern straying outside their area of responsibility.

Joint concerns
-Relying on active participation from people with diabetes Some HCPs and people with diabetes had concerns about proposals which might rely on active participation from people with diabetes, for example, the proposal for the person with diabetes to tick off a checklist when they have consented to/attended to screening. In the survey, a larger proportion of people with diabetes than HCPs agreed providing a checklist would be acceptable (69.5% and 35.3%, respectively) and feasible (75% and 35.3%, respectively). In the people with diabetes only and combined meeting, some people with diabetes felt having a checklist would help people be "proactive" in the management of their diabetes, while others thought that this would put too much responsibility on the person who "might lose or forget it". Some of those in the HCP only meeting thought that only motivated and engaged patients would use the checklist. -Balancing Act: Informing but not scaring people with diabetes Participants from all three meetings were concerned about achieving the balance between communicating the risks of diabetic retinopathy while not scaring people when informing them about screening. This concern related to several proposals to use other people with diabetes or HCPs to deliver messages. In the survey, both HCPs and people with diabetes agreed it would be acceptable to use a message from someone who has retinopathy and tells them it is important to go to screening before it is too late, there may be no symptoms and everyone with diabetes is at risk. However, 83.3% of people with diabetes agreed it would be feasible compared to 64.7% of HCPs. Participants across all meetings believed that "scaremongering" or "shock tactics" would not encourage people to attend. Rather than "shock" people, messages should inform them of the "truth" about the possible consequences of non-attendance and be provided "by the right person, in the right way". Both people with diabetes and HCPs agreed that the same message (tells them it is important to go to screening before it is too late, there may be no symptoms and everyone with diabetes is at risk) when delivered by HCP rather than another person with diabetes would be acceptable (100% and 88.2%, respectively) and feasible (91.7% and 82.3%, respectively). Participants in the people with diabetes only meeting thought the GP would be the best person to deliver a message to attend screening as people "trust" their GP and are "much more inclined to listen to them". HCPs in the HCP only and combined meeting had concerns that that delivering these messages during consultations would take a considerable amount of time.

Health care professionals' concerns -Resource implications
Concerns about the resource implications of delivering intervention proposals including time, staff, and money, were raised throughout all three meetings. Resource concerns were often a reason for HCPs' lesser agreement with proposals, especially those which aimed to encourage practice staff to ensure the person attends. Few people with diabetes and HCPs thought the proposal to provide a new resource to the practice (e.g., researcher checks if person registered, consented and/or attended) was feasible (53.8% vs 58.8%, respectively). While both agreed the proposal to prompt practice to check the (DRS) register during consultation and register person if necessary was acceptable, a slightly lower proportion of HCPs thought it was feasible (82.4% and 70.6%, respectively). They emphasized not having time for multiple prompts and reminders like letters or emails; "we absolutely don't have the time. We can't take anything on, it's just beyond unbelievable." -Motivating practice staff to make sure the person attends screening HCPs in both meetings had concerns about proposals to tell practices about the benefits/consequences of their patients attending/not attending. This was reflected in the different proportions who agreed such was feasible (people with diabetes 90.9% vs HCP 70.6%, respectively). Some HCPs believed financial incentives might be best to motivate GPs to ensure their patients are registered and attend DRS. HCPs in the combined meeting suggested that once practices have a registration uptake at a particular level, they could receive financial remuneration and therefore be "incentivised to do it (register patients)". There were also concerns about using feedback to motivate HCPs to encourage patients to attend, namely by providing practices with comparison numbers (% people attending in other practices/ nationally). This discussion arose around the proposal to give feedback on national or international uptake or targets. Some participants in the HCP only meeting felt "you would totally tap into [competitive] personalities" but there was a lack of consensus on this proposal in the combined meeting. Some participants in this meeting thought a comparator could be a useful motivator, whereas one GP noted that the differing demographic of patients across practices would make comparisons difficult. HCPs in both meetings argued that feedback needs to be specific and tailored to their practice and their patients, as national averages and practice comparisons are "totally useless" as they "cannot address that on a one-to-one level with a patient".
-Straying outside their area of responsibility As previously mentioned, the proposal to arrange practical support like transportation was not considered feasible by people with diabetes nor HCPs (58.3% vs 41.2%, respectively). HCPs in the HCP only and combined meeting felt this proposal strayed outside of their area of responsibility, as they mostly interpreted it as having to arrange the transportation for the patient themselves, something they felt was "not their (HCP) problem" as patients "need to take ownership and responsibility".

People with diabetes' concerns -Risking patient privacy
Participants in the people with diabetes only meeting were concerned that some proposals threatened their privacy. For example, arranging practical or social support would make it difficult for those who wish to keep their diabetes private to do so. Both people with diabetes and HCPs thought the proposal to provide a new resource to the practice like a researcher was not feasible (53.8% and 58.8%, respectively). A few participants in the people with diabetes only meeting were concerned about privacy should someone within the practice other than their GP/PN have access to their information. Contrastingly, more people with diabetes than HCPs thought this proposal would be acceptable (83.3% vs 64.7% respectively). However, this may be explained by HCP concerns about resourcing this proposal.

Comparing stakeholders' suggestions for new intervention content
Participants in the people with diabetes only meeting made 26 suggestions for new intervention content, of which 7 were deemed feasible to incorporate into the final intervention (30%). Participants in the combined meeting also made 26 new suggestions, of which 3 were feasible (15%). Participants in the HCP only meeting made 32 new suggestions, of which 7 were feasible (22%). Table 2 shows the suggestions for new intervention content that were deemed feasible to incorporate. New Conflicting opinions -Either participants in one small group agreed but participants in another small group disagreed with the recommendation or participants in one small group agreed but later in the discussion participants in the same small group disagreed with the recommendation.*BCT identified in the new suggestions that was not present in the intervention proposals outlined in the survey. Abbreviations: HCP = Health care professional, BCT = behaviour change techniques suggestions were deemed unfeasible to incorporate into the intervention if they could not be implemented in the Irish general practice setting. For example, participants in all three meetings suggested that the reminder message should be delivered by professionals outside general practice, that the national screening programme could modify their processes to make it easier for people with diabetes to register and attend the service, and that national-level changes (e.g., media campaign to improve attendance, establishing a national diabetes register) should be introduced to increase screening attendance.
New suggestions deemed feasible to incorporate into the intervention mapped to 12 BCTs in the taxonomy (Table 2). There were four additional BCTs identified in the new suggestions that were not present in the intervention proposals outlined in the survey: goal setting (outcome), restructuring the physical environment, material incentive (behaviour) and punishment. Additional information on the BCTs identified is provided in Supplementary File 7 in the Extended data 18 .

Comparing end users' contributions to the final intervention
The final intervention included a practice briefing, audit and feedback with technical support, practice-endorsed reminders (delivered in person, by phone and letter) and an information leaflet targeting key attitudinal and knowledge barriers. The people with diabetes only meeting had 23/51 (45%) recommendations incorporated into the final intervention, of these 20 were feedback on the intervention proposals and three were new suggestions. The combined meeting had 19/49 (39%) recommendations incorporated into the final intervention, of these 17 were proposed and two were new suggestions. The HCP only meeting had 24/55 (44%) recommendations incorporated into the final intervention, of these 21 were proposed and three were new suggestions. Table 2 shows the new suggestions that were incorporated into the final intervention. All three meetings made new suggestions that were deemed feasible but not incorporated into the final intervention. These suggestions, along with the reasons for exclusion (based on the APEASE criteria), are outlined in Supplementary File 8 in the Extended data 18 .

Summary of main findings and links to existing literature
Although there is growing awareness in the literature that involving different intervention end-users in the development process may have a different impact on the final intervention developed 8,11,27 , to our knowledge, this is the first study to examine and compare in detail the contributions of different intervention end-users as part of a consensus approach to inform intervention development.
There were three main findings. Firstly, people with diabetes and HCPs had both shared and unique opinions about the acceptability and feasibility of some aspects of the proposed intervention content. Some opinions were shared by both end-users and were echoed throughout all three consensus meetings, for example that there is a limited understanding of the screening process, or that we should balance informing people without scaring them when communicating about screening. However, HCPs also had unique concerns related to their role as healthcare providers, while people with diabetes had their own concerns about intervention proposals which might risk their privacy. Such differences suggest that while there is a common ground when it comes to preferences for and concerns about intervention content, there are some aspects of the intervention which may be a greater priority for different end-users. Secondly, participants in all three meetings made suggestions for new intervention content which mapped to BCTs that were not present in the proposed intervention content however, participants in the combined meeting made less feasible suggestions as they could not be implemented in the Irish general practice setting. Finally, participants in all three meetings made recommendations that were incorporated into the final intervention. However, participants in the combined meeting had fewer recommendations incorporated than the other two meetings.
In the meetings involving people with diabetes only and HCPs only, respective groups had different opinions about the delivery of messages to attend screening e.g., who should deliver the message, when the message should be delivered, and what the message should contain. Those in the meeting of people with diabetes only tended to base their recommendations on what would be most acceptable to the person with diabetes. In contrast, participants in the HCP only meeting focused more on what was feasible from a resource perspective. These concerns are consistent with reports of increased workload and staff burnout in Irish general practice 28,29 . In addition, some HCPs perceived that certain intervention proposals would involve straying outside their area of responsibility. They tended to disagree with proposals which they equated to an extra job or responsibility, understandable given the increasing responsibilities in general practice for chronic disease management 30 . Future intervention developers should consider these different perspectives of respective end-users so that they may involve them in the development process in the most effective way.
On the other hand, participants in this study also had joint preferences for intervention content. Both HCPs and people with diabetes were conscious that while it was important to outline the seriousness of retinopathy, there is a need to strike a balance between informing but not scaring people about the screening process and potential disease consequences from non-attendance. This aligns with the body of literature on the use, or avoidance, of fear appeals to encourage preventative health behaviours, evidence which has demonstrated that providing information about possible negative consequences may prompt defensive responses 31 . For instance, one US study found that avoidance of cancer risk information was associated with lower participation in colorectal cancer screening 32 . During the consensus meetings, people with diabetes and HCPs had concerns about intervention content which might scare or frighten people, such as having a message delivered by someone who is visually impaired or prompting the person to feel regret.
Intervention developers should select behaviour change techniques that promote adaptive, rather than maladaptive behaviour, as suggested by a qualitative study of fear appeals as a method in behaviour change interventions 33 . These joint contributions by participants in our study offer a useful perspective to intervention developers about how end-users will receive communication, but also demonstrates there are instances where end-users can share priorities for intervention content.
Our findings indicate that end-user groups' contributions to the intervention development process can differ based on whether they are involved separately or simultaneously. Participants in the combined meeting of people with diabetes and HCPs made fewer feasible suggestions for new intervention content and fewer recommendations from this meeting were incorporated into the final intervention. This suggests their contributions may have been influenced by group composition. Our previous analysis of participants' experiences of taking part in the consensus meetings found that, although members of the combined meeting appeared to work together, during follow-up data collection both end-user groups held different views about what intervention proposals would and would not work 13 . Our aim was to elicit feedback on components that would target PWD and HCPs, but both HCPs and PwD that participated in the combined meeting were uncomfortable with asserting what the other end-user group should or should not do. To fill this void, participants went off task and made suggestions that were outside the scope of an intervention intended for primary care 13 .
In this study, one skilled facilitator who was partly involved in the wider intervention development process facilitated all consensus meetings. While this was helpful in contributing to consistency, it is also possible that group dynamic and discussion might have been different had a person with diabetes co-facilitated the meetings e.g. this co-facilitator might have supported people in the combined meeting to speak on occasions where participants felt uncomfortable, or it was difficult to reach consensus. As the meeting involved small group discussion, we found this helped people to be forthcoming about their experiences and views, particularly in the meeting with PWD only.
This current study alongside our previous analysis suggests that it may be useful to involve each end-user group, those who will deliver the intervention and the intended target population, separately rather than simultaneously in a consensus process to inform intervention development. When involving different end-users together in a consensus process, researchers should also consider facilitating these groups differently, paying special attention to acknowledge potentially unique views while also reaching consensus. Previous research has recognised the potential complexity of multi-stakeholder involvement, highlighting the need to manage group interactions, potential power imbalances and synthesising the views of different groups 34 . One approach which might have been useful in the context of our research and could be relevant to future work in this field, would be hold the separate stakeholder groups first to allow for independent discussion and feedback, followed by a combined group in which consolidated feedback may be compared and discussed.
By comparing different ways of involving end-users, we hope to provide useful consideration for future intervention development. However, our study is just one example; involving a small number of participants. There are many factors which have contributed to final intervention content. We cannot definitively assert that involving different types of end users together will yield different intervention content. The ideas incorporated into the final intervention were not solely influenced by the consensus process, as researchers held power to make these final decisions. Ideally, future studies, involving different interventions and subject matters, would explore and report their experiences with involving end users and how this may have influenced intervention content. This would build a clearer picture of the optimal way to involve different s takeholders in this process.

Strengths and limitations
This study has several strengths including the use of a mixed methods, explanatory sequential design. Consensus meeting data supported the quantitative analysis by providing explanations, where available, for different participant ratings provided in the survey. By integrating the two, we aimed to draw out new findings beyond the information gained from the separate results 35 . Fetters et al. have reported that such qualitative methods are often applied in order to explore reasons why a phenomenon occurs or to describe the nature of an individual's experience 17 . The involvement of PPI contributors is a further strength of this research. A PPI partner (GF) was involved in the SWAT throughout the duration of the study and is a named co-author on this publication. A separate PPI group were involved in the development of the materials sent to participants prior to the consensus meetings. Supplementary files 3.2. and 3.3 in the Extended data 18 show how the study invitation letter and survey were improved as result of PPI feedback. These improvements helped to ensure that materials were more accessible and acceptable to participants.
This study includes a number of limitations. Firstly, as this was a SWAT, the primary aim of the consensus meetings was to review and discuss proposals for intervention content for the host trial and not to explicitly compare end-user contributions 15 . While the semi-structured approach of the meetings allowed participants to discuss proposed intervention content and generate new ideas for such content, it made it difficult to compare end-user contributions as the content and nature of the discussions varied across meetings. For example, some groups did not discuss certain survey ratings and intervention proposals, and some groups discussed particular proposals in more detail than others. Although the decisions about intervention components in this study were shaped by the consensus meeting discussions, had we adopted a more structured approach, we recognise PWD and HCP could have engaged in a more deliberate dialogue around final intervention components.
An additional limitation is the absence of some key end-users from the consensus meetings. There were no people with type 2 diabetes available to participate in the combined meeting. Despite using a range of strategies to recruit a representative sample of people with diabetes, we encountered issues with participant availability when arranging the combined meeting. Existing research has established that people with type 1 and type 2 diabetes have different experiences when managing their condition and engaging with HCPs and health services 37-39 . Therefore, the involvement of people with type 2 diabetes in the combined meeting could have potentially changed the nature of the discussion and led to different recommendations. There was also a lack of involvement of practice administrators in the consensus meeting. Participants in the HCP only meeting suggested that practice administrators would be best placed to deliver the intervention. Involving them in the consensus meetings may have led to different recommendations as they play a key role in undertaking clerical duties to support delivery of care, and as gatekeepers, help to preserve boundaries of organisation and controlling access to the practice 40 . However, the literature finds they are often overlooked by policymakers, undervalued by GPs and patients and excluded from research 40 . Future research in general practice should consider involving practice administrators to ensure that all user voices are heard.
A final limitation was the lack of capture of non-verbal cues such as when participants nod in agreement or disagreement. As this SWAT looked to examine and compare agreement with proposed intervention content, such non-verbal data may have been useful. While non-verbal cues can offer rich data 41 and we may have been able to capture this through video recording of the meeting, it has also been found that the use of video-recording equipment during focus groups can inhibit participants' interaction 42 .

Implications
The results of this SWAT informed the development of the IDEAs intervention which has been tested as part of a pilot cluster randomised trial with a view to progressing to a definitive trial 14 . Involving end-users in decisions about planning and conducting health research, policy and services is gaining increasing momentum and as such, PPI is now required by many health research funders, journals, and research ethics committees 43,44 . However, evidence on the impact of PPI is largely based on anecdotal reflections from researchers and members of the public which are descriptive and selective 45 .
Numerous studies have called for planned and methodologically rigorous research to evaluate the impact of PPI on the research process [46][47][48] . In this study, people with diabetes were involved as participants in the consensus meetings and not throughout the design and conduct of the research as PPI contributors. However, their role discussing and making decisions about the intervention content and delivery is not dissimilar to the active role that PPI contributors have in the research process [49][50][51] . This SWAT provides evidence on the contribution of different end-users to the intervention development process and how different end-users can have different priorities for intervention content. While our study provides useful reflections for future intervention development using consensus processes, results should be interpreted with caution given this is just one example of involving stakeholders, and other factors may have influenced the final intervention content.
Nevertheless, the results of this study, coupled with the results of our analysis of participants' experiences of taking part in the three separate meetings to inform intervention development 13 , suggest that it may potentially be more acceptable and useful to involve patients/members of the public and HCPs separately when conducting PPI activities. When involving stakeholders together in PPI activities, alternative approaches to facilitation may need to be considered. Furthermore, as the process and impact of PPI is heavily dependent on the context in which it is being conducted, further research exploring the experiences and contributions of different end-users is needed, including an exploration of different facilitation models. This would enable all individuals interested in involving patients and members of the public in health research, policy, planning and development of health care to design and conduct more appropriate and effective user involvement 8,52 .

Conclusion
UK Medical Research Council guidance on the development and evaluation of complex interventions states that interventions should be developed with user involvement, drawing on existing evidence and appropriate theory 1 . However, there is limited evidence on what different intervention users contribute to the intervention development process and whether their contributions differ according to group composition. Our findings show that preferences and priorities for intervention content can differ across end-user groups, and that suggestions and recommendations for intervention content and design may also vary depending on whether users are involved simultaneously or separately. Considering these findings, attention should be paid to how end-users are involved in intervention development processes. This will stand to help researchers to design and conduct more appropriate user involvement, which in turn, could potentially improve intervention fit with the end-user's perceived needs.

Data availability
Underlying data The consensus meeting data are not publicly available due to limitations based on the ethical approval received and healthcare provider representatives in a trial implementing a intervention to improve retinal screening among people living with diabetes. I find it difficult to find problems with this paper. The introduction sets the stage, the methods outline the parent trial and goals of the study here. The mixed methods complement each other and integrate very well compared to many mixed methods trials which are often comprised of quantitative methods and qualitative methods which exist in stand alone different components. I was very much impressed with this work.

Is the work clearly and accurately presented and does it cite the current literature? Yes
Is the study design appropriate and is the work technically sound? Yes

If applicable, is the statistical analysis and its interpretation appropriate? Yes
Are all the source data underlying the results available to ensure full reproducibility? Partly Are the conclusions drawn adequately supported by the results? Yes however, have some suggestions for the conceptual development of the paper as well as one or two small suggestions for revisions.
In terms of conceptual development, I found the positioning of people with diabetes and healthcare professions (HCP) as different kinds of 'end-users' somewhat problematic. While HCP certainly have an important stake in discussions about healthcare interventions, people with diabetes would surely be, as Daya (2020) 1 has previously noted, the 'key players that everyone else has a stake in'. I wonder how this positioning might have influenced the facilitation process, e.g., how a lack of group clarity around this might make it hard for people with diabetes to articulate ideas in combined meetings. Further to this, the researchers themselves appeared to be an invisible power in the decision-making process, with the power to determine what ideas were feasible/not feasible to include in the intervention outside of intervention development meetings. I wonder what might be different if people with diabetes and HCP had they had the opportunity to engage more deliberately in a dialogue around these decisions and how this might have shaped the final intervention. The researchers also note that a 'male professor' facilitated meetings, but it is unclear who this person was in relation to the research team, or how this might have influenced the process. For example, what might be different if a person with diabetes on the research team had co-facilitated? How might this support people in the combined groups to talk through conflicting views?
In terms of methodology, I do not think that the involvement of PPI was 'blurred' in the study (p.16). I think it is important to clarify the distinction between 'participants in research' and 'PPI research collaborators or co-researchers'. People with diabetes who were participants in the research study were involved in developing the intervention and would not constitute PPI research collaborators or co-researchers. While study participants had some decision-making power over the intervention, they did not have any decision-making power over the research strategy or interpretation of data. This is distinct from PPI research involvement, which in this study appears to be one person with diabetes (GF), who contributed to the research strategy including initial discussions that informed the grant application, development of recruitment materials, recruitment of people with diabetes to the study, and review of a draft publication. It is also important not to overstate PPI research involvement in this study. While it is certainly valuable to have input from one person with diabetes in the research project, ideally people with lived experience would be involved in equal numbers throughout all stages of the research process to ensure research priorities and interpretations are relevant and resonant to those most impacted by research-informed policy and services.
In terms of interpretation of findings, while it is interesting to consider how separating or bringing people together might influence intervention development, I think findings need to be interpreted more cautiously. Participant numbers are small, and many factors may have contributed to the number of suggestions made or ideas incorporated into the final intervention -not the least of which is the researchers' power to make this final decision.
Finally, some small issues relate to the term 'diabetes only' in the abstract. I think it would be best that groups are not named by an illness, and person first language would be preferable, e.g., 'People with diabetes, or PWD only' meeting would seem a better description, particularly as the term 'HCP only' meeting is used. It would also be good to use the full term for 'BCT' in Table 2 to make this more convenient for the reader.
I hope that this review is useful to the research team and support them in furthering the important work they are doing in determining best practice approaches to the development of interventions with people with lived experience and other key stakeholder groups.
professionals working in general practice (i.e., audit, feedback, electronic prompts etc.). Therefore, during the consensus meetings, people with diabetes and healthcare professionals were asked to discuss patient-level and practice-level components that were to be implemented as part of the intervention. It was made clear at the outset of the meetings that the focus was both people with diabetes and HCPs. We agree that this positioning likely influenced how PWD (and HCPs) contributed during the combined meeting. This is reflected in the discussion on our previous analysis of both PWD and HCP experiences of taking part in the consensus meetings in lines 467-481. We also agree with your suggestion that the researchers were an 'invisible power' in the decision-making process, who influenced the final intervention and have added a paragraph to the discussion section to address this in line 538-549.
2) Regarding the 'male professor' who facilitated meetings: We have added additional details to the methods section to clarify the relationship between the male facilitator and the research team in lines 122-125. We have also added sentences to the discussion to consider how involving a person with diabetes as a co-facilitator may have impacted the process; lines 482-489.
3) In terms of PPI methodology, we have taken your point on board and agree with your suggestion to clarify the distinction between research participants and PPI contributors. We have re-written the sentences in question as following in lines 582-586. In terms of PPI involvement in this study, GF was involved throughout the research process as you have correctly pointed out. We also consulted with an existing PPI group on the design of the consensus meeting invitation letter, evidence summary and self-completion survey. We have added further information to the methods section to give a more accurate depiction of the role of PPI in this study; lines 221-232.

4)
In terms of cautious interpretation of findings, we have now included further description in the i) Discussion (lines 503-512) and ii) Study Implications (lines 588-591).

5)
Thank you for pointing out the typo in the abstract. This is most certainly an error, as we fully appreciate the importance of using terminology which ensures individuals are not identified by an illness and have endeavoured to use appropriate language throughout the study. We have now amended this to say 'PWD only' in the abstract methods. We have also amended Table 2 as per your suggestion.

Competing Interests:
The authors have no competing interests to disclose.